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SUMMARY REPORT: The Amount and Nature of Teacher Help Necessary for 

Optimum Achievement Through Use of Programed Learning Devices 



Background of Study 

In 195^, B. F. Skinner published, ”The Science of Learning and the 
Art of Teaching.” Since that time an amazing burst of activity has taken 
place in the study and application of Skinner's ideas concerning what has 
come to be known as programed learning. Educators have been motivated 
by the fact that programed learning may be a practical means for applying 
to the classroom learning principles which have been proven with animals 
in the learning laboratories. Researchers in psychology and education 
have been intrigued by the possibilities of experimentation in which the 
teaching variables for the first time can be controlled in a practical 
manner. Engineers have been stimulated to devise mechanical devices and 
computer-based instructional systems. Within a period of approximately 
five years, 122 programs were published in a variety of subject fields. 
The practical educator, however, had few guidelines by which he could 
implement the instruction of pupils through programed learning. 

Research on programed learning has been concerned with three types 
of problem— (l) factors concerning the programs themselves (size of step 
multiple choice or constructed answer, etc.), (2) characteristics of 
learners (differences in academic achievement, ages, etc.), and (3) 
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variations in methods of pupil responses to programs (by-passing, branch- 
ing, linear, etc,). Although these problems are stimulating subjects for 
research, they have not resulted in much guidance to the classroom teacher. 
The research study which is reported in the following pages was an attempt 
to provide some information as to how programed learning devices could be 
administered effectively to students in the usual classroom situation. 

The particular variable which was analyzed was the role of the teacher. 

Programed learning materials provide the stimuli for learning and 
also reinforce the students* responses. Therefore, the question arises, 
what then is the role of the teacher? If the program teaches, in the 
traditional sense, what is the teacher to do? The following study was 
designed to determine if quite different teacher roles made a difference 
in terms of the achievement of students working with programed learning 
materials. A related question is whether or not students can learn from 
a program if a competent teacher is not available. 

In the Fall of 196l, the Columbia Basin Administrators* Study Council 
agreed to cooperate with the College of Education at Washington State Uni- 
versity to study the role of the teacher for optimum learning with pro- 
gramed learning materials. A grant from the research funds of the State 
Superintendent of Public Instruction, Louis Bruno, made it possible to 
pursue the study. Several school districts of the Columbia Basin contrib- 
uted teaching time and classroom teaching facilities for the experiments. 

Experimental Procedures 

The design of the experiment was that five classrooms of ninth grade 
algebra students would study mathematics set theory for twenty-five 
minutes per day during the regular algebra period. Four of the classes 
utilized a program published by Science Research Associates, Modern 



Mathematics — Book 2, The fifth class taught by a teacher used lesson 
plans which paralleled the SRA program. Each of the four classes using 
the program was under the direction of a teacher acting in a different 
teaching role. Class I was taught by a teacher who served merely as a 

monitor seeing that order was maintained and that students followed 

instructions as they studied with the programs, but offering no help on 
the mathematics material. Class II was under the direction of a teacher 
who also gave no help on mathematics, but who indicated an interest in 
the material and the method by completing the program during the same 
time as the students. Class III was taught by a teacher who offered 
assistance by answering such questions as individual students raised as 
they worked through the program. Class IV learned by completing the 
program and also by having the teacher review each day basic concepts 
covered in the program. Class V was taught by the teacher who prepared 
lesson plans which paralleled the program and who utilized a lecture- 
recitation approach for a comparable twenty-five minute period each day 

until the subject had been covered. 

Two sets of five ninth-grade algebra classes participated in the 
study. One group of five classes were members of two Moses Lake junior 
high schools. The second group consisted of one class from each of five 
different school districts— Othello, Warden, Connell, Soap Lake, and 
Ephrata. Throughout the report, results are recorded as obtained either 
from the :, Moses Lake group" or the u Basin group.” 

In March of 1962, before spring vacation, the research staff met 
with participating teichers and principals to explain the experiment and 
go over procedures. The teachers who would teach the parallel lesson 
plans were selected because they had attended mathematics institutes for 
teachers, and because they were willing to undertake the additional 
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preparation entailed • The two teachers with this assignment were given 
lesson plan materials which followed the programs exactly and which had 
been prepared by the research staff. Students in both experiments— the 
five groups in Moses Lake and the five in the Basin— were given an 
achievement test in algebra as a basis for judging the equivalence of the 
groups in mathematics. The groups learning by means of programs were 
permitted to work at their own pace, except that each was limited to a 
learning session of twenty-five minutes per day. In the program groups, 
each student, when he completed the program, was allowed a class session 
to review what he had. learned. After the review period, he was given an 
achievement test covering the material presented in the program. The same 
achievement test was given the non-program groups at the conclusion of 
their study of set theory. This test is termed "final test" in the report 
of results which follows. 

After the experimental teaching period, each class resumed its study 
of algebra. During the last week of the semester, all groups were given 
a different form of the achievement test on Modern Mathematics as a 
posttest. 

The achievement of the different groups was judged on the bases of 
group mean scores and standard deviations of scores on the "final test." 
The same measures were used to compare results on the "posttests." Groups 
were also compared as to the time required to complete the program or to 
cover the lesson plans. A questionnaire concerning various aspects of 
administering programs to students was given to the participation teachers. 

Individual scores on the final tests were correlated with individual 
scores on the algebra achievement test. One question in the minds of the 
research staff was whether or not programed learning was closely related 
to achievement through other forms of academic learning. By computing 



these correlations for each group separately, the research staff hoped 
to determine whether or not differences among groups on the final test 
applied to all levels of academic ability. 
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Limitations 

1 . As us ual in experiments with classroom groups, a major limitation 
was that there was no way to control variables among teachers other than 
those which were part of the experimental design. Variables of this kind 
which may have affected results were: Differences in teacher attitudes 

toward programed learning; differences in knowledge of the subject matter 
of the experiment (set theory); differences in philosophy of education, 
particularly regarding the importance of academic achievement in mathe- 
matics; differences in general preparation for teaching mathematics. 

2. Another limitation of the study was the impossibility of con- 
trolling certain factors in the learning environment such as the time of 
day for the mathematics class, interpersonal relationships of pupils within 
the experimental classes, learning habits acquired by the students m the 
different classrooms up to the time of the experiment— late spring. 

3 . There is some question in the minds of the research staff as to 
the relevance of the algebra achievement test for testing equivalence of 
groups. The content tested in the algebra test did not appear to be very 
similar to that of the program. It was the most appropriate test avail- 
able, however. (This was also an advantage in that previous knowledge 

of pupils as to set theory was practically zero.) 

The two group 4*s were given more time to master the content 
than the other groups. It was therefore difficult to determine whether 
it was the additional time or the different method which accounted for 



d ; r?ferences. 
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5. There was no way to control incidental learning by students 
outside of class, although teachers reported that they found no evidence 

that such learning occurred. 

Results of the Study 

The programed text, Modern Mathematics- Book 2, was chosen for use 
in this study. An important reason for this choice was the fact that the 
subject matter covered in the program had not been taught to the students 
who were to be subjects of the experiments, at least to the knowledge of 
the cooperating teachers. To be sure that previous knowledge on the part 
of the subjects, obtained from sources unknown to the teachers, would not 
be a factor in the study, the staff prepared and administered a 20~item 
multiple-choice test over the subject matter to be covered in the program. 
Students in the classes who were to participate in the experiments were 

given the test shortly before the study was to begin. 

Each of the items consisted of four possible choices of an answer. 
Purely on the basis of chance, a student might be expected to guess 
correctly one time out of four. There were 20 items on t.ie tests, so a 
score of five correct could be expected purely on the basis of chance. 
Table 1 reports the means of the ten classes on this pretest over the 
experimental subject matter. The Table also shows the average deviations 
of each group from the chance mean, 5-00. None of these deviations was 

significant. 

Pretest on Algebra A chievement 

The research staff used the Cooperative Algebra Achievement Test 
as a measure of the extent to which the classes were equal to each other 
in their ability to perform on the experimental programs. It was assumed 
that the ability to achieve in the usual algebra subject matter was 
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GROUP MEM SCORES ON MODERN MATHEMATICS PRETEST 
AND DEVIATIONS FROM CHMCE MEANS 



1 



Group 


Mean Score 


X 




Moses Lake Experiment 




^0. 


5.29 


+ .29 


2m 


5.oo 


.00 


3m 


4.97 


r .03 


Ur 


4.88 


- .12 


5m 


5.04 


+ .04- 




Basin Experiment 




lb 


5.38 


+ .38 


2b 


5.00 


.00 


3b 


5.52 


+ .52 


% 


4.65 


- .35 


% 


5.90 


+ .90 



3 



* 
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closely realted to the ability to achieve on the new subject matter used 
in the program — algebra set theory. Means of each group within the two 
experiments are reported in Table 2. 
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After the experimental teaching plan was completed and all students 
in the study had been given the final test over the experimental subject 
matter, the research staff correlated scores on this final test with 
scores obtained on the algebra pretest. This correlation was calculated 
to determine whether or not achievement in the usual algebra subject 
matter was in fact closely related to achievement in algebra set theory. 
Table 3 shows the correlation for each group on the two tests — algebra 
achievement and the final test over the programed material. 

All correlations are positive and higher than *50 with the exception 
of group 3 b among the Basin schools. These correlations are considered 
to represent a significant degree of relationship between algebra achieve- 
ment and achievement on the program material. 

Correlations of I.Q. with Achievement on the Program Content 

Final test scores on the experimental subject matter were correlated 
also with available intelligence quotient scores. The I.Q. scores were 
obtained from the cumulative record folders of individual students in the 
study. I.Q. scores were not available for all students, and different 
tests had been used to obtain these scores. The correlations reported 
in Table 3 are thus only approximate tendencies, rather than precise 
measures of the groups in the study. Groups 1, 2 and 3 in the two experi- 
ments at Moses Lake and the Basin schools are combined for comparison pur- 
poses, the two group Vs are combined, as are the group 5* s « The thinking 
behind this procedure was that the first three groups in each experiment 
were all utilizing the programed text as their primary method for learning, 



TABLE 2 



MEAN SCORES OF GROUPS ON THE 
ALGEBRA ACHIEVEMENT TEST 




Group 



Mean Scores 



Moses Lake Experiment 



5m 



25.24 

25.60 

21.00 

20.6 

21.4 



Basin Experiment 



TABLE 3 



CORRELATION OF INDIVIDUAL STUDENTS 9 .. SCORES 
ON THE ALGEBRA AND FINAL TEST 



Group 



Moses Lake Experiment 



3m 

m 

5m 



.85 

.75 

.84 

.57 

.58 



Basin Experiment 



while the other two groups could be grouped because of their similar 
treatments in the experiment* As the data could only be regarded as 
crude general measures of intelligence, this grouping procedure seemed 

justified. 

Correlations reported in Table 4 appear to support the correlations 
obtained in the comparison of traditional algebra achievement and achieve- 
ment on the experimental subject matter. The research staff concluded 
that more refined study of the relationship of intelligence and achieve- 
ment on the experimental subject matter was not necessary , and probably 
the two criteria were interrelated. The achievement on the algebra test 
was then used as the criterion for judging equivalence of groups. 

Some of the literature on programed learning suggests that since a 
student is allowed to work at his own pace through a programed text of 
small, easy steps, that individual differences in ability among students 
will not be as great a factor in achievement as under traditional learning 
methods. The results reported in Table 3 do not support this hypothesis, 
because there appears to be approximately equal correlations between 
intelligence and achievement regardless of the degree of dependence on the 

programed text. 

Results on the Final Test 

Since there was evidence that a relationship existed between algebra 
achievement and achievement on the material used in the study, differences 
among mean algebra scores were tested by analysis of variance to determine 
if these differences were great enough to affect the final scores. Table 5 
indicates that significant differences among these scores were discovered. 

The establishment of these differences led to the utilization of 
analysis of covariance to determine the degree of achievement among groups 
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TABLE 4 

CORRELATIONS BETWEEN I.Q. AND FINAL 
TEST SCORES CALCULATED BY GROUPS 




Group 



V 



6l 



IV 



54 



I, II, III 



57 
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TABLE 5 

ANALYSIS OF VARIANCE: ALGEBRA PRETEST 



Source of 
Variation 


df 


Sum of Squares 


Mean Square 


Among Means 


9 


2022.36 


224.71 


Within Means 


250 


15578.95 


62.32 


Total 


259 




1701.31 





Notes : 

F = 224.71 = 3.60 

62.32 

F sig. at .05 = 1.98 
F sig. at .01 = 2.60 
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relative to the achievement on the algebra tests. Table 6 presents, in 
addition to the mean final scores of the groups, the corrected final score 
as calculated by analysis of covariance. Significant differences were then 
determined among the ten scores. Table 6 summarizes this data. 

Posttests 

Approximately one week prior to the terminating of the school year, 
posttests covering the program material were given all groups. The average 
time lapse between the final test and posttest varied among groups from 
about two to three and one-half weeks. The percentage of forgetting for 
each group was then calculated. Table 7 presents the mean group scores 
on the posttest, and time lapse, and the percent of forgettxng. 

It is interesting to note that the group 5 's, which were the classes 
not using the program, constituted the first and second highest percentages 
of forgetting. It should be noted that group 5 of the Moses Lake schools 
had less lapsed time than two out of the other three Basin groups, yet 
forgot more than these two groups. 

Dispersion Within Groups 

There is some question as to whether programed materials tend to allow 
students of differing abilities to achieve more nearly the same amount than 
do traditional classroom techniques, particularly if students working with 
programed materials are allowed to progress at their own rate. This 
characteristic of programed instruction should manifest itself in the 
amount of dispersion of scores in the final tests. Consequently, it was 
felt that variability among scores should be tested and groups compared 

on the basis of this variability. 

Standard deviations on final scores were compared on the basis of a. * 
Table 8 presents the standard deviations of all groups involved in the study 

‘Garrett; Statistics in Psychology and Education, p. 156-standard scores. 
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the .05 level than group 3b* 

Group 5 m is significantly higher at the .01 level than groups lb and 2b. 

Group is significantly higher at the .01 level than group l b , and significantly higher than 

2 b at the .05 level. 

Group V is significantly higher at the .01 level than group l b . 
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TABLE 7 

POSTTEST SCORES, TIME-LAPSE BETWEEN FINAL AND 
POSTTEST, AND PERCENT FORGOTTEN BY STUDENTS 



Group 


Mean Posttest 
Scores 


Time Lapse 


$ Forgotten 




Moses Lake 


5 Experiment 




In 


34.01 


21.69 days 


11.6 


2m 


36.94 


20.6 


6.5 


3m 


31.47 


18.7 


7.6 


4m 


29.56 


21.52 


14.8 


5m 


30.07 


17.85 


14.8 




Basin Experiment 




lb 


23.44 


22.40 days 


13.8 


2b 


27.64 


25.81 


0.00 


3b 


• • 


• • 


• • 


4b 


41.28 


14.62 


6.8 


5b 


23.57 


22.00 


24.8 
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TABLE 8 

STANDARD DEVIATIONS OF GROUP 
SCORES ON FINAL TEST 



Group 



Standard Deviation 



Moses Lake Experiment 



lm • 
2m • 

3m • 

• 

5m • 



. 10,51 
. 9.96 
. 9.^7 
. 7.35 
. 7.89 



Basin Experiment 




6.25 

6.65 

6.^5 

7.82 

10.01 



Notes : 

Significant Differences Within Each Experiment: 

There were no significant differences among the 
Moses Lake Groups. 

Group 5b was significantly different at the .05 
level from groups 1^, 2^ and 3b» was no ^ significantly 
different from 4b* 

Significant Differences Among Groups When Experiments Are 
Combined: 

Group lb was significantly different at the .05 
level from groups lm and 2 m . 

Group 2b was significantly different at the .05 
level from groups lm and 2 m . 

Group 3b was significantly different at the .05 
level from groups l m and 2 m . 
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The data on standard deviations within the Basin experiment shows an 
apparent relationship between the amount of teacher help provided and the 
amount of dispersion of scores. The greater amounts of direct teacher 
help given in the classes in groups 4 and 5 are associated with relatively 
large dispersion of scores. The differences between groups 1,2, and 3 
in the Basin experiment and group 5 were all significant at the .05 level. 

To this extent these data substantiate the suggestion in programed learn- 
ing literature that programed learning is associated with smaller dispersion 
of scores than learning which involves greater direction by teachers. The 
data obtained from the Moses Lake experiment does not support this 
conclusion. No differences are significant among classes in the Moses 
Lake study, and the tendency is the opposite from that described as char- 
acterizing the Basin experiment — classes which had greater amounts of 
teacher help had smaller amounts of dispersion of scores. 



Time .. olved in Study 

The average time each group worked on the experimental subject matter 
is compared with the final score and teacher time involved in Table 9. In 
general, there seems to be little relationship between the average amount 
of time a group took to finish the program material, and the mean final 
score of that group. For example, group l b invested 40$ less time than 
group 2^, yet achieved nearly as well. Group 5 teachers invested the same 
amount of time that students spent on the material. The group 4 times were 
calculated on the conservative estimate that the teachers utilized 12 
minutes each period in the process of reviewing and explaining the material 
that the students had covered in previous 25-minute periods with their 
programed texts. It is difficult to estimate the actual expenditure of 
teacher time in the group 3 situation, since the teacher activity was 



19 



TABLE 9 

TIME SPENT BY TEACHERS AND STUDENTS AND 
GROUP MEAN SCORES ON FINAL TEST 



Group 


Mean No. 

Hours 
per Group 


Mean Score 
Each Group 


Estimated No. 
Hours Teachers 
Spent 




Moses Lake Experiment 


m 


15.52 


38.48 


0.00 


2 

m 


15.86 


39.52 


0.00 


3 m 

m 


17.23 


34.06 


• • 


It- 

m 


21.39 


34.28 


7.00 


5 m 


15.30 


35.30 


15.30 




Basin Experiment 






12.0? 


27.20 


0.00 


2 b 


20.90 


27.63 


0.00 


3b 


21,39 


29.70 


• • 




24.38 


44.31 


8.00 


^b 


14.58 


31.33 


14.58 
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primarily that of answering questions students night have during their 
25 minute period with the program. 

Graphic Presentation of the Data 

Fig tires 1 and 2 present in graphic form the relative achievements 
on the Algebra Pretest, the Final Test on the experimental subject matter, 
and the Posttest of each participation class. In all cases the means 
represented by the bar heights are derived from raw scores and are not the 
corrected means obtained by analysis of covariance, as described above. 

The relative heights of the bars representing Final Test scores and Post- 
test scores for any one group are not indicative of the exact amount of 
forgetting for that group, because there were sixty items on the fifth 
Final Test and only 51 on the Posttest. The significance of the graphs 
lies in the comparative results among different groups. 

In Figure 1 the heights of the bars for the different groups follow 
the same pattern for all three tests. This is true in Figure 2 also, with 
the exception that Algebra Pretests for groups IV and I are not consistent 
with these patterns. 

Figures 3 and 4 graphically compare not only the mean tendency of the 
groups on the Final Test, but also the dispersion of scores on this test 
for each participation class. 

The Teacher Opinionnaire 

The replies of the teachers to an opinionnaire provided additional 
data concerning the experiments. 

One question in the opinionnaire dealt with the lesson plans provided 
the control group teachers. In response to a question as to their adequacy, 
both control group teachers stated that some points of the material would 
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Fig. I. — Mean Scores on the Algebra Pretest, Final Test, 
and Posttest of the Moses Lake Groups. 





Algebra Test 
Final Test 
Post Test 




Fig. 2.-- Mean Scores on the 
and Posttest of the Basin Groups. 
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Fig. 4.-- Distribution of Basin Group Scores on Final Test. 
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have been better understood from a text. If a text had been provided, the 
control groups might have achieved higher on the final test over the material 

than they actually scored. 

This study was an attempt to determine if teacher attitude as mani- 
fested in the classroom toward programed instruction would have an influ- 
ence on the student achievement on the program. This problem was studied 
mainly by an analysis of the roles played by the group 1 and group 2 
teachers in that the group 1 teacher was to act as a monitor only, supply- 
ing no positive attitude toward the program, and the group 2 teacher was 
to lend psychological support to the students by actually proceeding 
through the program in the class session with the students. Responses to 
the opinionnaire revealed another attitude role present— that of a teacher 
not as enthusiastic about programed learning as the others. The serious 
doubts he expressed in the opinionnaire dealt with such areas as student 
interest and motivation, value of programed instruction compared with 
traditional procedures, and value of the use of this material as a major 
teaching tool. He also reported that the students cheated constantly, and 
that they could see little value in this type of learning. He questioned 
the learning theory involved, and expressed doubt that he would pursue the 
area of programed learning further. Most important to this study is the 
possibility that the opinions and attitudes of this teacher influenced his 
students to some degree, just as the opinions found in other teachers influ- 
enced their students. This particular group under discussion achieved the 

lowest of all groups involved in the study. 

Generally, the teachers reported students enjoyed working with the 

program, but that it was the average and superior students that worked best 
with it. Both control group teachers stated that the ‘*no grade* policy 
(no grades were given the students over material covered in the study) had 
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a negative influence on motivation, while all except one of the teachers 
using programs stated motivation was generally high despite the policy. 

Competition was reported high among students using the program at 
the beginning of the study, but this quickly disappeared and students became 
less concerned with the progress of others. Additionally, program length, 
period length, physical facilities, level of program difficulty, and other 
aspects of the study were deemed appropriate by the majority of teachers. 

All teachers but one felt that they would like to use programed 
materials in the future in their teaching, and that it was a valuable 
teaching tool if used in proper context. All except one were strongly in 
favor of this type of experimentation in the schools and expressed a desire 
to see it continue. 



Conclusions and Implications 



The experiment in Moses Lake and the experiment in the Basin schools 
did not agree in all respects, but it is evident that students can learn 
certain types of subject matter as well from a program as they can from 
a well -prepared teacher. The program used for this research, in common 
with all programs, does not include all the learning instruction which 
would be desirable for pupils studying modern mathematics. The program 
provides the knowledge, but only a teacher can guide students in appropriate 
uses, or applications, of this knowledge. It does appear, however, that 
a well -designed and tested program can relieve the teacher of routine 
presentation of basic facts and skills. It is equally obvious that a skill- 
ful and well-prepared teacher can present basic facts as well as a program— 
if he chooses to spend his time in this activity. 

The study showed that it is possible to save significant amounts of 
teacher time without sacrificing achievement by using the program instead 
of the teacher to present basic mathematical concepts. 

One of the group 4 classes clearly was superior to all the other groups 
in the experiment. The other group 4 class was not different in performance 
from other groups. We can reason from these two results that teaching can 
make a substantial improvement in the learning which results from programed 
instruction, but that such improvement does not result merely from the addi- 
tional effort on the part of the teacher. In the group 4 classroom in 
which a clear superiority in mathematics achievement was attained the 
teacher systematically reviewed the concepts taught by the program. In 
the other group 4 class, the teacher used a question and answer procedure 
to supplement the program. Apparently the systematic instruction was 
advantageous and the informal recitation was not. 
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A flaw in experimental design developed at the stage when the posttests 
administered. The experiment had to be scheduled late in the school 
year, with the result that too little time elapsed between the final and 
posttests in the cases of different experimental subjects. Students utili- 
zing the program method worked at their own pace and some students finished 
considerably later than others. The result of this procedure was that some 
students had materially less time than others between final and posttests, 
with increased opportunity to remember more of the experimental subject 
matter. Also, some groups (for reasons which the staff cannot explain) 
required significantly more time than other groups to complete the program. 

In spite of these severe limitations, the results suggest the possibility 
that students forget less as a result of programed learning than from 
nonprogramed instruction. 

Some evidence suggests that the range of achievement scores among 
students who learn from programs will be less than the range of ocher 
learners. The data in the comparisons of the Basin groups indicate that 
the range of scores increases in proportion to the amount of teacher help 
given. The result was not significant in the same manner when the data of 

the Moses Lake groups were analyzed. 

The research staff arbitrarily imposes a 25-minute limitation on the 
programed learning sessions. This practice period proved to be about right 
for most students. Interest stayed at a high level until late in the 
experiment. 

Analysis of the opinionnaires filled out by the participation teachers 
shows that programed learning in the form of programed textbooks is easily 
assimilated into present school procedures. No special physical facilities 
or radical revisions of teaching methods were considered necessary. The 



opinionnaire indicated that a negative attitude towards programed learning 
on the part of the teacher may affect the achievement of the students. 

This suggestion from the opinionnaire analysis is not conclusive evidence 
of this tendency, but this finding suggests that programed learning should 

not be initiated over the objections of the staff. 

The main implication from these results seems to be that programed 
learning is as effective as what may be expected from other good teaching 
conducted over the same time period, at least with certain kinds of 
subject matter. Programed learning supplemented by systematic teacher 
instruction can result in a significantly superior level of achievement 



by learners of varying academic abilities. 
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The objective of this study was to determine what effect different 
teacher roles would have on achievement of students using programed learn- 
ing devices. 

Two sets of five classrooms of ninth-grade algebra students studied 
mathematics set theory for 25 minutes each day during the regular algebra 
period. In each set, four of the classes utilised the program, Mem 
Mathematics -Book 2. The fifth class was taught by a teacher using lesson 
plans which paralleled the program. Each of the four programed learning 
classes was taught by a teacher acting in a different teaching role. The 
four teaching conditions were: 

a. The teacher merely monitored the programed learning session. 

b. The teacher gave no help but completed the program himself. 

c. The teacher answered individual student® s questions. 

d. The teacher supplemented the program by reviewing basic concepts. 
St uden t achievement in the ten different classes was compared on the basis 
of mean scores and dispersions of scores for each group. Statistical 
treatment included both analyses of variance and covariance. 

In one set of five classes, no significant differences in student 
achievement were found. In the other set of five classes one of the expert 
mental classes achieved at a level which was significantly higher than any 
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of the other classes. In this class students combined study of the program 
with supplementation by the teacher. The fact that this type of teacher 
supplementation was superior in one experimental set and not in the other 
led to further study of the specific procedures used by the two teachers. 

The research staff tentatively concluded that it was the systematic nature 

of the teacher supplementation which was superior. 

In the procedure which was different and successful the teacher reviewed 

by orally summarizing each concept presented in the program in the order 
in which it occurredc In the other procedure, the teacher reviewed by 
means of answering students 3 questions in a class discussion. 

In one set of experimental classes, dispersion of scores on the achieve- 
ment test varied in direct proportion to the amount of teaching help given 
students. In the other set of five classes there were no significant 
differences in terms of dispersion, and such differences as occurred were 

in the opposite direction as in the other experiment. 

Teachers reported no serious problems in administering the program 
which was in the form of a programed text. Analysis of the opinionnaires 
suggested the possibility that a negative teacher attitude towards programed 
learning might be reflected in relatively low student achievement. 

A comparison of achievement and teacher time expended indicated that 
the programs resulted in a substantial saving in teacher time, with no loss 

in achievement. 

Programed learning achievement correlated positively with previous 
achievement in algebra, suggesting that the same student characteristics 
associated with success under traditional academic learning conditions 
would be associated with success under programed learning conditions. 

The research staff concluded that programed learning directed towards 
acnievement of mathematical concepts is as effective as other forms of 



instruction conducted within the same time span, and, administratively, 
programed learning can be used to save substantial amounts of teacher time 
Programed learning supplemented by systematic teacher instruction can 
result in a significantly superior level of achievement. 



